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S ABES 



SABES is the System for Adult Basic Education Support, a comprehensive training and technical assistance 
initiative for adult literacy educators and programs. Its goal is to strengthen and enhance literacy services 
and thus to enable adult learners to attain literacy skills. 

SABES accomplishes this goal through staff and program development workshops, consultation, mini- 
courses, mentoring and peer coaching, and other training activities provided by five Regional Support 
Centers located at community colleges throughout Massachusetts. SABES also offers a 15-hour Orientation 
that introduces new staff to adult education theory and practice and enables them to build support net- 
works. Visit us at our website: www.sabes.org 

SABES also maintains an adult literacy Clearinghouse to collect, evaluate, and disseminate ABE materi- 
als, curricula, methodologies, and program models, and encourages the development and use of practi- 
tioner and learner-generated materials. Each of the five SABES Regional Support Centers similarly offers pro- 
gram support and a lending library. SABES maintains an Adult Literacy Hotline, a statewide referral service 
which responds to calls from new learners and volunteers. The Hotline number is 1-800-447-8844. 

The SABES Central Resource Center, a program of World Education, publishes a statewide quarterly 
newsletter, “Field Notes” (formerly “Bright Ideas'*), and journals on topics of interest to adult literacy pro- 
fessionals, such as this volume of "Adventures in Assessment" 

The first three volumes of "Adventures in Assessment" present a comprehensive view of the state of 
practice in Massachusetts through articles written by adult literacy practitioners. Volume 1, Getting Started, 
includes start-up and intake activities; Volume 2, Ongoing, shares tools for ongoing assessment as part of 
the learning process; Volume 3, Looking Back, Starting Again, focuses on tools and procedures used at the 
end of a cycle or term, including self, class, and group evaluation by both teachers and learners. Volume 4 
covered a range of interests, and Volume 5, The Tale of the Tools is dedicated to reflecting on Component 
3 tools of alternative assessment. Volume 6, Responding to the Dream Conference, is dedicated to respons- 
es to Volumes 1-5. Volume 7, The Partnership Project, highlighted writings from a mentoring project for prac- 
titioners interested in learning about participatory assessment. Volumes 8-12 cover a range of topics, includ- 
ing education reform, workplace education, and learner involvement in assessment. Volume 13 focuses on 
issues related to building systems of accountability. Volume 14, Examining Performance , presents a range 
of articles that focus on performance-based assessment. 

We'd like to see your contribution. If you would like to submit an article for our next issue, contact 
Editor Marie Cora. 

Opinions expressed in "Adventures in Assessment" are those of the authors and not necessarily the 
opinions of SABES or its funders. 

Permission is granted to reproduce portions of this journal; we request, however, appropriate credit be 
given to the author and to "Adventures in Assessment" 

"Adventures in Assessment" is free to DOE-funded Massachusetts programs; out-of-state requests will 
be charged a nominal fee. Please write to, or call: 



Marie T. Cora, World Education, 

44 Farnsworth St., Boston, MA 02210-1211 
617-482-9485 • MCORA@WORLDED.ORG 
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Introduction. Volume 15: Assessment in Motion 



BY MARIE T. CORA 



t he articles in Volume 15 are 
a reflection of the past year’s 
continued work on building 
an accountability system for the 
state. Some writers contribute 
their experience learning to use newly 
mandated assessments, others write about 
their efforts to make the new assessment 
requirements fit their programs. There are 
also articles describing current assessment 
resources, and ones discussing the impor- 
tance of the goal-setting process for the 
Massachusetts system. I think it’s fair 
to say that all these are examples of 
“assessment in motion” — reflections 
of literacy workers who are in the midst 
of change in the world of assessment. 

Roger Hooper writes about how the 
Massachusetts Distance Learning Projects 
are managing the new state assessment 
requirements. His article finishes with 
some reflective questions including 
whether these programs should develop 
an assessment suited to the uniqueness 
of the Distance Learning experience. 

Dianna Baycich and Tim Ponder write 
about the National Institute for Literacy’s 
Special Collection for Assessment. This 
website is an invaluable resource for 
practitioners looking for information and 
resources on issues in ABE assessment. 
There are an additional 10 topic areas 
covered in the Special Collections, all well 
worth the trip. 



Two short pieces appear in this 
volume: one that advertises SABES’ 
newest on-line resource, the Assessment 
Support Website launched in October of 
2002. The other short article informs us 
about the Goal Setting Work Group 
convened by ACLS this spring, whose 
mission is to improve processes and 
tools for setting goals with students. 

Last year, Massachusetts adopted the 
REEP Writing Assessment as one of the 
state’s required tests. Folks from the 
Arlington, VA program who developed the 
REEP assessment contributed an article 
about the test and its scoring rubric in 
Volume 14 of Adventures in Assessment. 

In this volume, Luanne Teller writes 
about her program’s experience learning 
to use this test; she also outlines for us 
what they find particularly useful about 
REEP, and some challenges with the 
assessment that they feel could be 
improved upon. 

The BEST Test was also adopted as a 
state required assessment last year. Also 
at that time, field testing for the new 
computerized version of the test (called 
BEST Plus) was beginning. Massachusetts 
practitioners were involved with this field 
test. We first hear from Carol Van Duzer, 
who describes the new BEST Plus for the 
reader; and then from Susan Arida, who 
was a participant in the field test here 
in Massachusetts. 
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INTRODUCTION 



Finally, an annotated bibliography 
of Adventures in Assessment articles 
that focus on placement and goal-setting 
is included. This resource was developed 
last fall to help practitioners find materials 
they could use for placing students into 
classes and for setting goals with them. 

Last spring, Massachusetts was 
at the beginning of a phase of change 
in our assessment landscape. Since the 



fall, practitioners and programs have been 
in the midst of learning and doing. 

What we are experiencing now is hands- 
on and sometimes trial by fire: we are 
in motion. 

Your thoughts and ideas are welcomed 
and encouraged. If you would like 
to submit an article or have comments, 
please feel free to contact me at 
mcom@worlded.orQ 
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Assessment Challenges in Supported Distance Learning 
How the ABE Distance Learning Project is Implementing 
The Massachusetts ABE Assessment Policies and Procedures 



BY ROGER HOOPER 



t he Massachusetts ABE Distance 
Learning Project is a statewide 
collaborative including three 
specially funded demonstration 
pilot sites, and statewide project 
coordination staff. Our vision of “supported 
distance learning” (SDL) includes delivery 
of content and instruction largely through 
technology and multimedia curriculum 
packages, but also includes learners in 
some face to face contact with teachers, 
counselors, tutors and other students for 
support and assessment. The integration 
of various supports into a distance learning 
delivery system reflects the commonly 
understood needs of all adult basic 
education learners. 

These learners, who bring so much wis- 
dom and life experience to the educational 
process, often cannot bring the tools we 
expect of the typical “student” such as 
time management, understanding of their 
own learning needs and styles, ability to 
prioritize and evaluate the material to be 
learned, and the skill to navigate the barri- 
ers that work and family often present to 
the adult learner. Adult basic education 
learners need supports to work with these 
academic and social issues. In classroom 
ABE programs, teachers and counselors 
assist the learner “on-site” with these 
issues when necessary. But learners in 
our SDL pilot programs, although similar 
in strengths and needs to all other ABE 
students, do not have the same level 



of immediate or frequent access to 
teachers and counselors to address these 
issues. Our Pilot Programs provide access 
to necessary supports for learners that 
can help ensure their retention and 
progress even while the major part of 
their learning takes place at a distance 
from the program site and the staff. 

The mission of the Project is to 
research which supports best help adult 
learners to succeed in distance learning, 
to document models of effective SDL pro- 
gramming, and to recommend by the end 
of FY’04 a schedule of program quality 
indicators and cost factors for SDL. The 
Massachusetts Department of Education 
has committed to evaluating our recom- 
mendations prior to deciding whether to 
offer SDL among the program options 
fundable to all eligible programs under 
the next state multi-year ABE funding cycle. 

Our Project agenda for FY’03 includes 
building effective strategies to assess 
learners, and evaluate program effective- 
ness in SDL. We are building these on 
the platform of services that our demon- 
stration pilot sites have developed since 
the Project’s inception four years ago. 

The framework for this task is the 
Massachusetts Department of Education’s 
Assessment Policy for ABE. The challenge 
is to create strategies that work within 
the specialized services developed in our 
pilot sites, and that also meet the overall 
requirements of that Policy. 



"Our vision of “supported 
distance learning" (SDL) 
includes delivery 
of content and 
instruction largely 
through technology 
and multimedia 
curriculum packages, 
but also includes 
learners in some face 
to face contact with 
teachers, counselors, 
tutors and other 
students for support 
and assessment." 





ASSESSMENT CHALLENGES IN SUPPORTED DISTANCE LEARNING 



"In OUT broad SDL The Agenda for ABE Distance Learning 



developmental project, 
assessment is an 
“umbrella" under which 
we include measuring 



The Massachusetts Adult Basic 
Education Distance Learning Project 
entered its fourth year of operations in 
March of 2003. The Project has made 
strides in a broad spectrum of areas of 
program development for SDL including: 



participation, retention, 
progress and completion 
of goals." 



identifying and using technology 
appropriate and accessible to ABE 
learners who enroll in SDL programs; 
finding and implementing published 
multimedia curriculum packages that 
conform to the body of knowledge 
needed by ABE learners in SDL (and in 
traditional programs) — in both ESOL 
and in ASE/Pre-GED instruction; 



• developing regional “hubs” through 
collaboration with other agencies and 
programs, doing outreach and recruit- 
ment of learners, and creating training 
and orientation models to bring these 
learners into the SDL program; 



Overall, assessment is a larger 
concept than just testing learners. 

In our broad SDL developmental project, 
it is an “umbrella” under which we 
include measuring participation, retention, 
progress and completion of goals. The 
Massachusetts ABE Assessment Policy 
focuses on the assessment of learners’ 
level gain as established by standardized 
testing instruments. 

Since our SDL pilot programs are 
included under this policy, the pilot sites 
are developing logistics to try and meet 
the state demands. Because the policy 
was designed to be implemented in a 
classroom-based learning program (stu- 
dents in classrooms with a teacher and a 
regular schedule of meetings over a spec- 
ified cycle of instruction), it is a challenge 
to develop strategies that make it fit SDL 
programs in which our learners 

• mostly (but not exclusively) work off- 
site and independently with multime- 
dia curriculum packages; 



• developing and establishing within an 
SDL context supports for learners that 
support retention and educational 
progress; 

• sustaining SDL pilot program staff with 
ongoing technical in-service support, 
and sharing lessons learned about 
program and staff development for 
SDL with the larger community of ABE 
practitioners; 

• and exploring strategies for ASE/Pre- 
GED and ESOL learner assessment in 
SDL contexts. 



• rarely work in groups; 

• have a tendency toward “open 
entry/open exit” participation even if 
the services are designed in set 
instructional cycles; 

• and only occasionally meet face to 
face with a trained, professional ABE 
practitioner. 

A Variety of Assessment Issues 

In addition to implementing the State 
policy for assessment of learner level and 
learning gain, our SDL development agen- 
da includes other assessment issues: 
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ASSESSMENT CHALLENGES IN SUPPORTED DISTANCE LEARNING 



• how to include SDL program planning 
needs in the State’s Guidelines for 
Writing a Comprehensive Assessment 
of a Community’s Needs and Assets; 

• how to identify and assess practition- 
ers’ skills needed in SDL services in 
administration, instruction, counseling, 
technology “literacy,” etc.; 

• how to assess learners for readiness 
and pre-service training/orientation 
needs as they prepare to enroll in SDL 
instruction (including assessment of 
learning styles and technology skills); 

• how to assess learner participation in 
the SDL services, when “attendance” 
cannot be measured as “seat time”. 

Implementing the 2002 State 
Assessment Policy 

Initially, the Pilot Sites developed 
assessment strategies as follows: 

ESOL Assessment 

The ESOL pilot sites are structured 
around use of CROSSROADS CAF£ 
(Intelecom and Heinle and Heinle 
Publishers, 1997; includes video, work- 
texts, photostory readers, partners’ guide, 
assessment package, and teacher’s 
guide). These sites have created tools 
including an intake/assessment instru- 
ment with specific references to CROSS- 
ROADS CAf£ content for in-house place- 
ment. 

For reporting progress and learning 
gain to meet state and national reporting 
requirements, the SDL programs initially 
administered the commercial assessments 
used in the parent agency’s core class- 



room program (either the BEST Test or 
the ESLOA). 

Of the two pilots using ESOL assess- 
ments, only one was using the BEST prior 
to FY’03, but still found it necessary to 
put staff through the statewide training 
and certification procedures needed to 
use BEST validly under the new state 
policy. The other pilot previously used the 
ESLOA, and thus needed to switch to the 
BEST Test and meet the required staff 
training. Also, both of these demonstra- 
tion sites as well as their host agencies 
needed training on the REEP assessment. 

ASE/Pre-GED 

Initially, only one of the three pilot sites 
was focused on ASE/Pre-GED instruction 
instead of ESOL. Before the end of the 
second year of the project, however, both 
the other Pilot Sites began to include 
ASE/Pre-GED learners in their service plan, 
and all were implementing an appropriate 
multimedia curriculum package (WORK- 
PLACE ESSENTIAL SKILLS; developed and 
published in 1999 by PBS and KET-TV, 
currently distributed entirely through KET- 
TV; includes video, print, web, internet, 
and teacher guides). Currently, the pilot 
programs are also using (to some degree) 
TV411 (ALMA) and GED Connection (KET- 
TV/PBS) each of which as a similar range 
of component media resources. 

Prior to the 2002 assessment policies, 
the sites either used the AMES or the 
ABLE to measure learning gain. Upon 
publication of the Policy, all three Pilot 
Sites converted to TABE and sent staff to 
train on this instrument as required by 
DOE. 

Intake/placement tools were developed 
for learners in the ASE/Pre-GED programs 
as well. These include important 
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ASSESSMENT CHALLENGES IN SUPPORTED DISTANCE LEARNING 



1 Note that this work to create 
a systematic, reliable protocol 
to measure "participation** in 
distance learning in ways to incor- 
porate time spent in independent , 
multimedia-supported learning, is 
motivated at least 
in part by the needs of NRS 
reporting requirements, and that 
the national consortium of states 
working with the Project IDEAL 
staff at the University of Michigan 
to establish standards and criteria 
for distance learning for adult 
learners is equally involved in this 
effort. Massachusetts is one of the 
thirteen states currently partici- 
pating in Project IDEAL. 
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resources to assess the learner’s readi- 
ness for computer-based distance learning 
in terms of both computer literacy and 
independent learning skills. An orientation 
for new distance learners following intake 
offers a chance for them to relate their 
technical skills to the specific learning 
program, or to develop skills which are 
assessed as weak or missing at intake. 
After this orientation, learners are 
transitioned into active instructional use 
of Workplace Essential Skills in the SDL 
program. 

For Further Discussion 

Below is a sample of questions, which 
might deserve further discussion and 
study. 

l. How do we best determine when 

and how frequently assessments 

are administered to SDL learners? 

The state policy refers to hours of 
instruction as seat time or attendance in 
class. However, in our Pilot Programs, we 
are counting participation by our SDL 
learners in a variety of ways, including 
independent learning supported by video, 
print, web, and CD-ROM/Software materi- 
als. The SDL learners do spend some 
time in direct face-to-face contact with 
teachers, and that might be considered 
the practical equivalent to attendance or 
seat time in a classroom setting. However, 
that's only part of the picture. In addition, 
our SDL learners receive instructional sup- 
port at a distance, through (for example) 
mail, e-mail, and telephone. And even 
beyond that, these learners spend at 
least (on average) four times as many 
hours in independent learning “at a 



distance” supported by the materials 
mentioned above, than they spend receiv- 
ing support from an instructor either face 
to face or at a distance. We are establish- 
ing a unique protocol for assessing par- 
ticipation and intensity of services based 
on this specialized approach to counting 
instructional intensity in SDL 1 . Policy for 
timing of administering assessments 
should reflect the unique criteria used 
to measure participation and intensity of 
services in SDL. 

2. What it the best strategy for making the 
initial assessment a low anxiety non- 
threatening event for a newly enrolling 
SDL learner? 

The expectation in classroom-based 
programs is that the learner may need to 
become familiarized with the teacher, the 
class, and the expectations and proce- 
dures of the course before taking a test 
that will directly impact their education as 
an adult. For classroom programs meeting 
regularly several days a week, this is a 
readily manageable timing issue. But SDL 
offers a different challenge. Currently in 
the SDL programs, the learner may meet 
the teacher in an intake/enrollment ses- 
sion. The second meeting may be the ori- 
entation to the SDL program, which can 
involve from 2 to 15 hours of orientation 
and training, depending on the program 
and the learner. The next subsequent 
meeting may be several days or even 
weeks later, when the learner has com- 
pleted an initial phase of independent 
‘supported distance learning.' At this 
meeting, the learner meets with his/her 
instructor to get support, orientation, 
and clarification of any problems. It may 
be difficult to identify any underlying 
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ASSESSMENT CHALLENGES IN SUPPORTED DISTANCE LEARNING 



process of familiarization and environ- 
mental comfort-building for the learner in 
this context, which is dramatically differ- 
ent than the experience in the classroom. 
At what point in this extended time frame 
does it make sense to “mark” a point in 
time when the learner will be familiar 
enough with the teacher and the services 
to be comfortable taking the initial 
assessment? 

3. How well does the assessment instru- 
ment used for supported distance learn- 
ers “match” the contents and scope of 
their instructional program? 

In Massachusetts, the statewide adop- 
tion of a uniform, single set of assess- 
ment tools for ABE and ESOL learners 
overall required a team of practitioners 
and assessment specialists to "match" 
the standardized tests being evaluated 
to the Massachusetts ABE Curriculum 
Frameworks. The goal was to locate, or 
if necessary, develop a standardized 
assessment tool that actually tests the 
knowledge and skills outlined in the state 
Curriculum Frameworks. However, in our 
SDL pilot programs, the overall scope and 
content of the instructional program is 
defined not simply by the state 
Frameworks but also by the scope and 
sequence of the commercially published 
multimedia curriculum packages that our 
supported distance learner work with. 
Since as much as 80% of the supported 
distance learners' participation in the edu- 
cational program occurs with the use of 
these commercial multimedia materials 
(print, video, software, web, internet), 
should there be a specific effort to assess 
the "content match" between the assess- 
ment and the multimedia curriculum used 



in SDL programs, as well as with the con- "However, there are 
tent and scope of the state Frameworks 

used overall in Massachusetts ABE? already Some indications 



4. Can a strategy for “ distance 
assessment" for SDL programs be 
developed that will accommodate the 
standards and policy of the M A DOE 
Assessment Guidelines? 

Each of our three pilot sites in MA 
is doing assessment “by the book” 
according to the state policy. That means 
that each learner enrolling in SDL services 
must take the REEP, the BEST or the 
TABE assessment on site, with directed 
supervision of a trained practitioner. 
However, there are already some 
indications that some of the very factors 
that “qualify” an adult learner for SDL 
services versus classroom services may 
make this on-site, direct assessment 
a challenge. Those factors include a lack 
of transportation or child care, unsuitable 
work schedules, even physical handicaps 
that make it equally as difficult for 
a learner to come for an assessment 
as it would for them to come to class. 

The current practice is to find ways to 
overcome those barriers and bring the 
leaner into the program site for an 
assessment. But it is reasonable to 
look ahead and project a scenario where 
distance assessment (in some format yet 
undeveloped) will be as important to the 
leaner as the SDL program on instruction 
and learner support itself. We have no 
doubt that this is a high hurdle to reach, 
and that it is something for the long 
range rather than the current or emerging 
status of SDL, but it remains on the 
horizon for further exploration and 
perhaps developmental effort. 



that some of the very 
factors that "qualify" 
an adult learner for 
SDL services versus 
classroom services 
may make this on-site, 
direct assessment 
a challenge." 
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FAIR ASSESSMENT PRACTICES GIVING STUDENTS EQUITABLE OPPORTUNITIES TO DEMONSTRATE LEARNING 



Roger Hooper manages the Massachusetts 
ABE Distance Learning Project through the 
Adult Literacy Resource institute in the 
Graduate College of Education, University 
of Massachusetts/Boston, He coordinates 
the Project Partnership that includes 
SABES, Massachusetts DOE, and three 
Demonstration Pilot Sites based in the 
Northern Berkshires (Massachusetts 
College of Liberal Arts), greater Springfield 
(international Language Institute with 
Corporation for Public Management) and 
Cape Cod (Cape Cod Community College 
ACCCESS Program ). 
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The NIFL LINCS Assessment Special Collection 



I INCS is the Literacy Information 
and Communication System, 
a project of the National Institute 
for Literacy (NIFL). LINCS 
combines a variety 

of resources including online discussions, 
technology training, and an extensive 
database of literacy materials. One of the 
key components of LINCS are the Special 
Collections. The LINCS Special Collections 
are peer-reviewed, content-oriented 
collections of high-quality literacy 
practices and materials for use in adult 
education and literacy programs. There 
are currently 11 topics covered in the 
Collections including English as a Second 
Language, Equipped for the Future, 
Literacy and Learning Disabilities, and 
Policy and Legislation. The homepage 
for the LINCS Special Collections can be 
found at h ttp :/M/ww. n ifL go v/incs/co llec- 
tions/collections.htmL 

The Assessment Special Collection 
(http://literacy.kent.edu/Midwest/assess- 
ment) launched in June of 2001 and 
funded in part by the Office of Adult 
and Vocational Education, provides indi- 
viduals dealing with assessment and 
evaluation in adult literacy programs 
a means to access a variety of resources 
electronically. This Collection focuses 
on assessment and evaluation topics 
for both program and instruction. 
Professional development topics and 
current event information dealing with 
assessment are also included. 



Sites for the Assessment Special 
Collection are found through recommen- 
dations by practitioners, postings on dis- 
cussion lists, articles in various profes- 
sional journals and newsletters, and web 
searches. Once sites are identified they 
are sent to a Core Knowledge Group 
(CKG) who decide if they are suitable for 
the Special Collection. The members of 
the CKG are considered experts in the 
field of assessment in adult education 
and currently represent the following 
agencies: Institute for the Study of Adult 
Literacy, Penn State University; Ohio 
Literacy Resource Center; Stark County 
Even Start, Ohio; SABES/World Education, 
MA; Arkansas Adult Learning Resource 
Center; National Center for Family 
Literacy; Partnership for Family Education 
and Support, Louisville, KY; and LIFT, 
Missouri Literacy Resource Center. 
Members of the CKG work in pairs to 
evaluate the sites based on the quality 
and usefulness of the information and the 
organization of the site. If the two review- 
ers do not agree, the site is sent to a 
third member of the CKG for review. New 
sites are added to the Special Collection 
several times a year. CKG members com- 
municate by e-mail and periodic confer- 
ence calls. The CKG also plans meetings 
at national conferences when several of 
them will be attending. 

In the Assessment Special Collection 
you will find high quality resources 
with information about various aspects 
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BY DIANNA BAYCICH 
AND TIM PONDER 

"The LINCS Special 
Collections are 
peer-reviewed, 
content-oriented 
collections of high- 
quality literacy 
practices and 
materials for use 
in adult education 
and literacy 
programs." 
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THE NIFL LINCS ASSESSMENT SPECIAL COLLECTION 



of assessment and evaluation. Currently 
the target audiences are Teachers/Tutors 
and Managers/Administrators. We are 
planning to add information for students 
in the near future. A new focus of the 
collection addresses resources for state 
staff. Working with the National Adult 
Education Professional Development 
Consortium and a small group of state 
staff from across the country, quality 
assessment information, models, and best 
practices relevant to state staff are being 
identified and reviewed for inclusion in 
this section to be launched in early 2003. 

Since the Assessment Special 
Collection is relatively new, we are always 
happy to receive site recommendations. 

If you have sites you would like to 
recommend or have comments or 
suggestions about the site you can 
contact Dianna Baycich at dbavcich@liter- 
acy.kent.edu or Tim Ponder at 
tponder@zhost.net 



Dianna Baycich has been working at the 
Ohio Literacy Resource Center for three 
years , and in Adult Education since 1991. 
She has a Master's degree in Adult 
Learning and Development and is currently 
working on her Ph.D. Dianna lives with 6 
cats. 

Tim Ponder is Assistant Director in 
charge of NIFL projects at the Ohio Literacy 
Resource Center. His current responsibili- 
ties include overseeing the Midwest 
Regional Technology Center 
and both the Family Literacy and 
Assessment Special Collection projects . Tim 
has worked with various state and national 
adult literacy organizations 
providing technical support and training in 
the area of technology. 
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ASSESSMENT SUPPORT WEBSITE 



Did you know...? 

SABES has created an on-line resource to help you 
with state assessment requirements 



--TJsabes Assessment Support Web Site - Microsoft Internet Explorer proyided^ylb^l^^MHS 
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Go to www.sabes.org/assessment 



Testing requirements in brief 
Information on the BEST, TABE, 
and REEP Assessments 
Helpful links 



Adventures in Assessment on-line 
Assessment glossary 
Bibliographies 

And much more so visit often 



For additional information and questions, contact Marie Cora at MCora@worlded.org . 
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Did You Know?. 
About the Goal Setting Work Group? 



a 05 (Adult and Community 
Learning Services) has 
convened a group of eight 
practitioners to develop 
guidelines for setting goals with students. 
The Massachusetts ABE system strives to 
put student goals at its center. Programs 
and practitioners set goals with students 
in a variety of ways. It is the purpose 
of the Goal Setting Work Group to 
strengthen this work by identifying the 
field’s best practices and by developing 
guidelines for goal setting processes. 

The activities the group is engaging 
in over the course of the four month 
project include: 

• Surveying program directors 

• Collecting sample lesson plans for 
goal setting with students 

• Conducting a literature review 

The survey revealed that many practi- 
tioners see the goal setting process as 
separate from curriculum and instruction. 
Practitioners also felt that there is 
presently a dual system of goal setting: 
one that is useful for the teacher and 
student, and one that is useful for record- 
ing data in the SMARTT system. The Work 
Group is developing resources to bridge 
the curriculum gap by helping 
practitioners see goal setting as 



integrated with instruction. They also 
intend to propose processes that will 
enable goal setting to serve both the 
purposes of the teacher and student, and 
of the SMARTT system. 

The survey also asked people what 
challenges they faced when trying to 
set goals with students, and recommen- 
dations they have for overcoming these 
challenges. The Group intends to make 
recommendations regarding these 
challenges to ACLS. 

From the review of the literature on 
goal setting, the group intends to develop 
guidelines for good goal-setting processes 
that programs can use as a resource tool. 

One of the things that the Group feels 
is important is that practitioners are sure 
on the purpose of goal setting. Goals can 
be used for a variety of purposes includ- 
ing for the program, for students, or for 
funders. The Group believes that goal 
setting needs to be embedded in a 
particular context in order for those 
goals to be collected effectively, and 
ultimately, for them to be useful. 

This is an exciting project that should 
help practitioners better identify students’ 
goals, and learn about ways that the 
process can be useful to inform instruc- 
tion. The Group plans to present their 
findings and recommendations, along 
with sample lesson plans for goal setting, 
later this spring. We’re looking forward 
to it! 
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"The purpose of the 
Goal Setting Work 
Group is to . . . identify 
the field's best 
practices and develop 
guidelines for goal 
setting processes." 
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Making Sense of REEP 



was prepared to be annoyed. That’s 
right: just what I needed —another 
acronym to add to my list! Another 
mandate to add to my list! A beau- 
tiful summer day lost to training. 
Just what I needed. Boy, was l prepared 
to be annoyed. 

Imagine my dismay. I like REEP. I have 
embraced REEP. REEP is making my 
program better. Imagine! 

When we first learned that new 
assessment policies were coming, we were 
in just our second year of funding. As a 
new program, we struggled with finding 
"reliable" and "valid" assessment tools. 
We had great intake tools, and we were 
meeting all of DOE’s reporting require- 
ments, but we sensed we could do better. 

During this time, we received 
a curriculum grant which we used to 
develop thematic units. We wanted to 
move away from standardized testing 
towards performance-based assessments. 
Our staff, experienced with student 
portfolio work, believed that performance- 
based assessments would allow us to 
more accurately assess student levels, 
enhance instructor feedback, and better 
track student progress. As we proceeded, 
we remained sensitive to concerns about 
objectivity, continuity over time and 
different instructors, and the amount of 
staff time required to evaluate work and 
prepare feedback for students. 

We held many staff meetings, 
attended workshops, and conducted 




research. The task of assembling 
a comprehensive list of competencies was 
not nearly as daunting as the challenge 
of somehow converting or “crosswalking” 
it to an SPL (Student Performance Level) 
number that was "useable" for DOE 
reporting purposes. It has always been 
my management style to make data-driv- 
en decisions. I prefer to plan for program 
improvement based on what l “know I 
know” instead of what I “think I know,” 
and what I knew for sure was that there 
had to be a better way. 

Thank goodness for the work that 
ACLS, SABES, and the PAWG (Performance 
Accountability Work Group) did to 
facilitate our understanding about this 
necessary response to NRS and federal 
mandates. Over the course of the 
previous year, the workshops on 
Standardized Assessments, the PAWG 
updates , NRS Requirements , and Using 
Data for Program Improvement were 
immeasurably helpful in preparing for 
what lay ahead — and more importantly 
— slowly but surely bringing my staff 
up to speed. Thankfully, we were able 
to immediately skip right over the 
“reactionary mode” phase directly into 
“implementation mode.” We felt 
adequately prepared and informed, 
and we fully understood this was a policy 
that was going to have to be followed. 

So, we decided to make the best of it. 
(Yes, my staff is beginning to be known 
as the “lemonade” squad.) 



ADVENTURES IN ASSESSMENT 



BY LUANNE TELLER 

"Our staff, experienced 
with student portfolio 
work, believed that 
performance-based 
assessments would 
allow us to more 
accurately assess 
student levels, 
enhance instructor 
feedback, and better 
track student 
progress." 
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"Ultimately, we have 
found REEP to be 
efficient, minimally 
intrusive to 
instructional time, 
and more reliable 
and consistent than 
other tools we had 
used." 



"We have begun placing 
students in "writing 
workshops" according 
to their REEP scores." 



Ultimately, we have found REEP 
to be efficient, minimally intrusive to 
instructional time, and more reliable and 
consistent than other tools we had used. 

In fact, the information we are able to 
glean from student papers is well worth 
the time required to administer and score 
the tests. It frees us from trying to convert 
performance-based assessments to an SPL 
number, while providing detailed feedback 
on students’ strengths and weaknesses. 

We have begun placing students in 
“writing workshops” according to their 
REEP scores. This enables instructors to 
focus on more targeted instruction and 
also helps solve the “leveling” problem 
for students whose oral skills are 
significantly higher or lower than their 
reading/writing abilities. Most students 
have responded with enthusiasm, and feel 
like it’s a “special” opportunity to be in 
these writing workshops/seminars. 

Here is what we have learned from our 
first two rounds of REEP testing: 

• We were somewhat surprised at the 
amount of commonality across papers. 
We are able to identify several areas 
of common writing errors which is 
helpful to classroom instruction. 

• tt seems obvious, but it bears repeat- 
ing that some of our students are very 
gifted writers — a talent which has lit- 
tle to do with their ability to communi- 
cate in English. Even some of our most 
beginning writers are able to show 
"voice” and tremendous "spirit" in 
their writing. 

• We value the level of scoring objectivity 
that comes from not having previously 
worked with the students. For example, 



scorers were not able to understand 
one student’s writing about surgery 
on her hand. After the test was 
scored, we shared it with the 
instructor who was immediately able 
to derive the meaning because she 
knew the student had just had hand 
surgery. We feel the “disconnect” 
between the scorers and the students 
provides more reliable information. 

• We have learned to be very specific 
in explaining to students how their 
work is scored and kept. In our post 
September 11^ world, many of our 
students were cautious about replying 
to the prompt, “Write one bad thing 
about your life here.” Students were 
afraid that we would ‘Turn their 
papers over to the government.” It’s 
easy to understand why they would 
be reluctant to write anything bad 
about America. While I initially 
supported the idea of using “regional 
scorers,” I now feel we need to be 
sensitive to these very valid concerns. 
How might writing be impacted if 
students felt that “outsiders” would 
have access to their papers? 

• It really is important to “recalibrate” 
each time we administer the assess- 
ment. We find the anchor papers 
especially helpful in this process. 

Some of our concerns/questions about 

the REEP include: 

• We wish the rubric were more specific 
with greater detail for each level. 

• Overall, we have found the REEP 

to be an accurate reflection of student 
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MAKING SENSE OF REEP 



abilities and levels, however there is 
the "on any given day" aspect that 
applies to any standardized test. We 
were recently surprised to see one of 
our more advanced students score sig- 
nificantly lower on his second REEP 
test than on his first, only to later 
learn that there were extenuating cir- 
cumstances. There is nothing we can 
do about this: we can’t retest and the 
score must stand. We can only hope 
that his third assessment will be more 
reflective of his true abilities. 

• The use of REEP as the official assess- 
ment in higher ESOL levels has given 
some students the impression that we 
— as a system — value writing more 
than oral communication. This is the 
exact opposite of what the majority of 
our higher level ESOL students want, 
which is to concentrate on pronuncia- 
tion and speech. 

• At our REEP training, we were told 
that a typical learner gain is .4 each 
year. If this is accurate, then it may be 
much more difficult to show learner 
gain based on federal SPL levels. 

• In fact, after just two administrations 
of the REEP, we are somewhat 
surprised by the relatively small gains 
that students show, especially in the 
higher levels. It seems much harder 
to progress from a REEP level 5 to 

a level 6, than from a REEP level 1 
to a level 2. Our Site Coordinator, 
an ESOL specialist, estimates that the 
year-end assessments will show 
greater gain. Students learning new 
material will require more practice 
before becoming confident enough 



to take new risks in their writing. I'm 
told that a little more time and a little 
more practice will work wonders. 

Here is what we plan to do next: 

• We are keenly aware of pending fiscal 
constraints, but how we would love 
another curriculum grant! While some 
have voiced concerns about the time 
required to implement the new assess- 
ments, we feel more acutely the lack 
of planning time to develop a strategy 
in response to what the assessments 
tell us. 

• We would like to analyze and review 
the competencies/benchmarks 
established within our curriculum units 
and align them more closely with the 
REEP rubric levels. Since we are elect- 
ing to work with students in writing 
groups based on their REEP levels, 

it will help instructors target instruc- 
tion based on student portfolio and 
assessment writing. Of equal value, 
it will also provide students with 
clear information on their progress, 
including what they still need to mas- 
ter in order to move to the next level. 
We had begun work on these check- 
lists, but found it difficult to convert 
them into SPLs. With the new 
assessment policies, we are free 
to develop them in a way that meets 
our instructional and student learning 
needs without concern for how they 
could be used for reporting purposes. 

When asked to write about our 

experiences with the REEP Writing 

Assessment, I was quick to reply that 

on most days, we feel like we have more 
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"It really is important 
to "recalibrate" each 
time we administer 
the assessment." 



PAGE 19 VOLUME 15 



MAKING SENSE OF REEP 



questions than answers, and that we 
are an ever-evolving work in progress. 

I am thankful for a dedicated staff that 
embraces new ideas, and always looks 
to how they can better serve our 
students. Our Community Partnership 
remains steadfast in their support and 
encouragement in response to our chang- 
ing needs. I am grateful for the help of 
my peers and never cease to be amazed 



by the level of support among ABE 
practitioners. Last, but not least, we are 
always open to new ideas. We are eager 
to hear from other programs, and hope 
you will share your experiences with us! 



Luanne Teller is the Director of the 
Stoughton ABE Program at Massasoit 
Community College . 
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The BEST Plus - A New Way to Assess Oral English Skills 



Massachusetts was invited by the Center for Applied Linguistics to participate in the field 
test of the new BEST Plus assessment that will be available in FY2004 . Eight instructors 
from six programs were trained to administer the computer-assisted test 
All participants were experienced adult education ESOL instructors , expressed interest 
in receiving the training , and had experience administering either the original BEST Test , 
or another oral assessment for ESOL . Each program tested 60 students within 
3 weeks of the training session. 



n this era of accountability, 
programs serving adult English 
language learners have been 
searching for an assessment 
tool that meets accountability 
requirements, can be administered in 
a reasonable amount of time, and 
provides feedback on learner progress and 
information that will assist in improving 
program quality. The BEST Plus responds 
to these needs. 

Developed by the Center for Applied 
Linguistics (CAL), the BEST Plus is an 
individually administered face-to-face 
scripted oral interview designed to assess 
the English language proficiency of adult 
English language learners. The assessment 
is an adaptation of the Basic English 
Skills Test (BEST) Oral Interview that 
was developed during the early 1980s. 

Like the BEST, the BEST Plus assesses 
interpersonal conversational communica- 
tion using every day language. It is able 
to assess a wider range of English lan- 
guage proficiency levels than the original 
BEST — from Student Performance Level 
(SPL) 0 (no ability whatsoever) to SPL X 



(native-like proficiency). The BEST Plus 
integrates current knowledge in the 
assessment of speaking and listening 
skills with the latest knowledge and 
practice in educational measurement 
to present a precise and flexible oral 
assessment. The adaptive nature of the 
BEST Plus means that examinees will 
receive different tasks each time they 
take the test, particularly as their skills 
improve and they are administered more 
challenging items. 

The BEST Plus comes in two versions 
— a computer adaptive assessment 
on CD or a semi-adaptive print-based 
version. The BEST Plus begins with 
"warm-up" items to make the examinee 
feel comfortable conversing with the test 
administrator. These include personal 
information questions that are commonly 
asked of English language learners (e.g.. 
What languages do you speak?). The 
examinees are then administered several 
sets of thematically related questions on 
topics such as health, recreation, and 
education. 
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"The adaptive nature 
of the BEST Plus 
means that examinees 
will receive different 
tasks each time 
they take the test, 
particularly as their 
skills improve and 
they are administered 
more challenging 
items." 
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THE BEST PLUS - A NEW WAY TO ASSESS ORAL ENGLISH SKILLS 



In the computer-adaptive version, the 
test items are delivered via a computer. 
The test administrator asks the examinee 
a question presented on the computer 
screen, listens to the examinee's 
response, determines the score for that 
item, and enters the score into the com- 
puter. The computer then selects the 
next test item, choosing items most 
appropriate for the examinee according 
to the scores entered for each response. 
The only time the examinee sees the 
computer screen is if the test item 
involves a picture. At the end of each 
interview, the computer automatically 
generates a score report. This report 
includes the examinee’s scaled score, SPL 
Level, National Reporting System (NRS) 
level, and sub-scores for listening com- 
prehension, language complexity, and 
communication. The time it takes to 
administer this version of the test ranges 
from 5-15 minutes, depending upon the 
ability level of the examinee. Generally, 
examinees with higher language proficien- 
cy have more to say, increasing the test- 
ing time. 

In the print-based version, a quick 
locator test determines the level of test 
items (1, 2, or 3) that will most efficiently 
determine the functioning level of the 
examinee. These items are arranged 
in fixed-form level tests. The test 
administrator asks the examinee each 
item on the level test, scores the items, 
and marks the score on a score sheet. 
When the test is completed, the test 
administrator can total up the score to 
receive an estimate of the examinee’s 
proficiency. A more precise level can be 
determined by entering the score data 
into a simple computer program that 
generates the same score report as 
the computer adaptive version. The 



print-based version of the test takes 
approximately 10-12 minutes to administer. 
There are three forms of the print-based 
version so that pre- and post- tests can 
be administered without compromising 
the validity of the test. 

For both the computer-adaptive and 
print-based versions, the test administra- 
tor uses the same rubric to guide scoring 
decisions. Each examinee response is 
scored for three aspects of language: 

Listening comprehension refers to how 
well the examinee understood the question. 

Language complexity refers to how well 
the examinee organized and elaborated 
the response. 

Communication refers to how clearly 
the examinee communicated meaning. 

As the first computer-adaptive oral 
language assessment for adult English 
language learners, the BEST Plus is 
breaking new ground. At the same time, 
the BEST Plus is responsive to the needs 
of the field for an assessment tool that 
can link learner progress to accountability 
purposes, produce diagnostic information, 
be administered quickly, remain easy to 
score, and generate a new version of test 
questions each time it is administered. 

The BEST Plus will be available 
from the Center for Applied Linguistics 
in late March 2003. For additional infor- 
mation, visit the BEST Plus web page 
at http:/Mww.cal.ora/BEST/compbest.htm . 



Carol Van Duzer is the Training and 
Technical Assistance Coordinator 
at the National Center for ESL Literacy 
Education (NCLE) in Washington , DC . 
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The BEST Plus at YMCA 



a s an instructor-participant in 
the BEST Plus pilot program, 

I am delighted to say that the 
BEST Plus just may be the 
assessment that has been missing from 
my life. I know that I am not the only 
adult educator who has been disappointed 
by the absence of an assessment method 
that could elicit data useful to learner, 
instructor and administrator . . . and (at 
the risk of sounding awfully demanding) 
be efficient, effective and fun. But the 
work that has been done with the BEST 
to earn the “plus” does give one reason 
to hope. 

At the YMCA of Greater Boston’s 
International Learning Center, I am an 
instructor, counselor and coordinator in 
a program that serves about 250 adult 
learners of diverse linguistic, educational 
and ethnic backgrounds. Our classes, 
offered morning, afternoon and evening 
are funded in several ways including 
a grant from the Massachusetts 
Department of Education. 

My experience with the BEST Plus 
includes on-site testing of 65 students 
representing proficiency levels 1-8 in July 
2002. This was preceded in June by a 
two-day training led by the professionals 
from CAL who had developed the test. 
And then in December 2002, I was asked 
to score videotaped BEST Plus assess- 
ment conversations with 30 students. 

Though my experience is limited to 
three sessions, it seems to me that the 



questions that make-up the BEST Plus 
respect and engage the learner. The 
prompts are modeled on conversations 
that might actually occur between two 
people from different cultures. The text, 
which must be followed exactly, is not 
stiff or condescending. The computerized 
program includes prompts of varying 
complexity, and has the ability to filter 
the test questions based on the answers 
given by the learner, so not every 
question associated with a particular 
subject needs to be asked. This 
eliminates embarrassment and unneces- 
sary stress for the person being tested. 
This feature, while still providing data 
on a learner’s language limitations, also 
gives the learner an appreciation for what 
he or she can do. 

Some learners were reluctant to 
participate at first, but once the grapevine 
started humming, students actually began 
to ask to participate. They enjoyed the 
experience, which is in sharp contrast 
to the reaction most students had 
to the BEST without the Plus. Unlike 
its prototype, the BEST Plus’s computer 
format presents colorful, contemporary 
and culturally diverse images and 
situations as lead-ins to conversations 
on a variety of subjects familiar in daily 
American life. 

Nothing surprised me more, however, 
than my own enjoyment of the process. 

I expected to be bored and at the same 
time anxious about my ability to score 
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“The BEST Plus’s 
computer format 
presents colorful, 
contemporary and 
culturally diverse 
images and situations 
as lead-ins to 
conversations on 
a variety of subjects 
familiar in daily 
American life." 
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THE BEST PLUS AT YMCA 



"The BEST Plus also 
includes a range 
of subjects with 
questions 
of corresponding 
complexity so that 
the test can examine 
a particular skill 
with a variety 
of prompts, which 
make the testing 
more interesting 
and therefore easier 
for the test 
administrator." 



consistently and accurately. But the 
training had provided a good base for 
evaluating learner responses and though 
the testing required a good energy level, 
it was not nearly as exhausting as the 
BEST because it was interesting to both 
the learner and the tester. The BEST Plus 
also includes a range of subjects with 
questions of corresponding complexity 
so that the test can examine a particular 
skill with a variety of prompts, which 
make the testing more interesting and 
therefore easier for the test administrator. 
Once accustomed to the scoring system, 
the program simplifies data input and 
offers a greater scoring range; it also 
allows simultaneous scoring of listening 
comprehension, language complexity, and 
communication. My biggest challenge was 
finding a space that had computer access 
and afforded the necessary privacy. The 
right space is essential for consistent 
scoring and to maintain a comfortable 
mood so that the student can maintain 
his or her focus. 

At this point, I can only speak 
about the experience of testing with the 
BEST Plus and what I observed among 
students during the testing. I have no 
way of determining its efficacy in 
providing data to funding agencies, 
program administrators and policymakers. 
We did not receive any feedback on 
student scores so I do not know if the 



scores squared with the learner's 
class placement, nor do I know if my 
interpretation of the test directives was 
consistent with other testers. I only know 
that my data was submitted and received. 
I also had a list of perhaps 10 or 15 
points that I felt needed to be addressed. 
For example if the test is to be consistent 
and administered with exact adherence 
to the text, then idiomatic phrases that 
might have regional usage should either 
be eliminated, or like phrases common 
to a variety of locales should be included. 
Most of my comments did not address 
philosophical or contextual issues, but 
easily adjustable details. 

I am interested in participating in the 
pilot again. I took on the project last 
summer because I actually had the time 
to do it. Our program director, lonela 
Istrate sympathized with our frustration 
over mandated assessments without 
an appropriate assessment tool, and 
we always like the idea of trying new 
materials and new strategies. We were 
also curious about what the “Plus” might 
mean. We found that it is for the BEST, 
a positive addition. 



Susan Arida wears many hats at the YMCA, 
International Learning Center 
of Greater Boston. 
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The BEST Plus at El Centro del Cardenal 



BY ALEXANDRA 
S U LI KOWS K I 



started working for El Centro del 
Cardenal in May 2000. I was in 
charge of the whole intake process 
for our ESL classes, and since my 
first day, I have been looking for 
ways of improving our current assessment 
procedures. 1 even developed assessment 
tools for the Massachusetts Curriculum 
Frameworks that would reflect more 
accurately our students’ proficiency levels 
in both their initial intake and progress 
evaluations. My problem, of course, 
was to get something that would be 
“standard” with all other programs 
and recognized statewide. 

I continued using the BEST Test 
because it was a requirement, but I have 
to admit that it wasn’t my “favorite.” 

Last year I heard about the BEST 
Plus pilot and I was very excited about 
participating. 

In our program, 65 students 
participated in the field test. I tried 
to get candidates that would represent 
a variety of English proficiency levels, 
cultural backgrounds, ages, and gender, 
so that the final results could be 
representative. It took approximately 
three weeks to complete the pilot. 

I interviewed some newcomers 
and also some of our current students 
in order to get some feedback on their 
experience with the new BEST Plus. 

Those who were familiar with the 
original BEST Test liked the new test 
better. Because the test is set up as 




a conversation with me helped them 
to feel more relaxed, and so they 
participated more. 

I liked the BEST Plus because it gave 
the students more “real” everyday topics 
to talk about like health, education, 
employment, recreation, travel, 
community — things they could relate 
to their own experience. The computer- 
based version was really appealing 
to the participants and to me since 
the exam took less time to administer, 
it automatically generated a score, and 
the pictures were colorful and easy to 
describe. 

I also like the fact that language 
proficiency was divided into three 
categories: Listening comprehension, 
language complexity, and communication. 
The scoring rubrics described more 
accurately the students’ levels and, in 
most cases, made clear-cut boundaries 
among the more advanced students. This 
aspect is something that we did not have 
before with the original BEST Test. 

There is also an “adaptive” version 
of the BEST Plus, in which the computer 
selects the next appropriate test item 
automatically, adapting to the student’s 
skills. I didn’t get the chance to work 
with the adaptive version and 
consequently, for the lower levels, 
some of the questions were too 
challenging, and sometimes discouraging 
for the participants. I am anxiously 
awaiting the final, adaptive version. 



"I liked the BEST Plus 
because it gave the 
students more "real" 
everyday topics 
to talk about like 
health, education, 
employment, 
recreation, travel, 
community - things 
they could relate to 
their own experience." 

"The scoring rubrics 
described more 
accurately the 
students' levels and, 
in most cases, made 
clear-cut boundaries 
among the more 
advanced students." 
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I was very happy to be able to 
participate in this pilot. I am really 
looking forward to seeing the final 
version. I hope this meets our needs 
for better assessment and, consequently, 
better program quality. 



Alexandra Sulikowski is a program 
counselor in the HABLE Program (Hispanic 
Adults Basic Literacy in English for 
Education and Employment) at El Centro 
del Cardenal in the South End 
of Boston. 
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Adventures in Assessment 
Briefly annotated bibliography of articles focusing 
on ln-take r Placement, and Goal-Setting 



BY MARIE T. CORA 

Massachusetts has been building a system of accountability in ABE since January 2001. 

Our system attempts to place the needs of adult students at the center 

of our programs. Thus , helping adult students identify their own needs — or goals — 

becomes the first priority in the system. 

Elsewhere in this volume , you have read about the Goal Setting Work Group 
convened by ACLS (Adult and Community Learning Services , MA DOE), it is their intent to 
adopt, or if necessary develop , a uniform goal-setting process for all 
programs to use across the state, so as to reach the goal of a more effective and equi- 
table service delivery system. 

Over the past 2 years, Massachusetts has found that the initial phases of enrolling 
a learner in our programs includes determining goals, and placing the student 
in a class appropriate for those goals, and appropriate to his or her level of literacy. 

Adventures in Assessment has a wealth of information and resources in the areas 
of placement and goal-setting. Below are all the articles in Volumes 1 through 14 that 
focus on placement and goal-setting. 



Volume 1: Getting Started 

All of these articles are relevant 
in terms of setting goals and discussing 
tools and procedures for in-take and/or 
placement of adult students. 

“Assessment Issues: Research and Practice” 
by Lauren McGrail: Provides a brief 
overview of alternative assessment tools 
for in-take, in-class, and end of cycle. 

“Partners in Evaluation” by Johan Uvin: 
Primarily about program evaluation, but 
conducted by participants; good ideas 
for ways to determine what adult students 
think. 



“Getting in Touch: Participants” Goals 
and Issues" by Lucille Fandel: Nice 
presentation of goal-setting activities. 

“Read/Write/Now Adult Learning Center 
Assessment Adventures” by Janet Kelly: 
Discusses alternative assessment 
procedures in detail, including tools 
for initial contact; screening and 
placement; goals-setting; and 
progress. 

“Down and Dirty Miscue Analysis” 
by Lindy Whiton: Miscue analysis 
described; this is useful as an additional 
placement and/or diagnostic tool. 
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“The Education Goals Assessment Packet” 
by Martha Germanowski: Setting goals 
in detail. 

“Alternative Assessment: An Annotated 
Bibliography” by Don Robishaw, ed.: 

Brief overview of alternative assessment 
tools for goal-setting and progress. 

Appendix: Samples and examples 
of actual tools. 

Volume 2: Ongoing 

All articles focus on ongoing 
assessment; there are many 
self-assessment tools described. 

“Self-Assessment: Doing and Reflecting” 
by Paul Trunnel: Describes self-assess- 
ment tools useful for goals-setting. 

“Three by Three by Four: Ongoing 
Assessment at the Community 
Learning Center” by Karen Ebbitt et al.: 
Goals-setting tools and procedures 
described. 

Appendix: Samples of tools for setting 
goals and surveying skills. 

Volume 3: Looking Back, Starting 
Again 

The first section focuses primarily 
on reflecting on progress at the end of 
a cycle; the second section focuses on 
the cyclic nature of using final evaluation 
to inform the beginning of a new cycle. 

“Assessment and Planning: Giving 
Students Ownership” by Amy Gluckman et 
al. Discusses processes of in-take and the 



review of goals previously set by adult 
students. 

“The Whole-Person Approach in Math 
Assessment" by Mary Jane Schmitt and 
Helen Jones. Math assessment in detail 
including learner comfort; long and short 
term goals; and initial assessment. 

Appendix: Samples of some tools for 
both initial and on-going assessment. 

Volume 4 

This volume covers several "getting 
started" and "ongoing" articles. 

“Group Goal Setting Activities” by PECE 
Resource and Planning Guide: Provides 
specific tools and procedures for 
conducting goals-setting with a group. 

“Empowering the Student Through Goal- 
Setting” by Susan Martin et al.: Describes 
goals-setting procedures including initial 
assessment, in-take, and on-going meas- 
urement. 

“The Informal Reading Inventory” by 
Eileen Barry: This tool is useful as an 
additional placement and/or diagnostic 
tool (higher level). 

“The ESL Classroom as Community: How 
Self Assessment Can Work” by Dulany 
Alexander: Useful procedure for aiding the 
goal-setting process. 

Volume 5: Tale of the Tools 

This volume focuses entirely on 
tools for use in on-going assessment; 
although not the focus of this annotated 
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bibliography, I highly recommend this 
volume: the tools described are excellent. 
This article, however, is relevant to this 
document: 

“The Case for Pre-Goal Setting” 
by Don Robishaw: Goal-setting is 
described as a strategy as opposed 
to a static activity; interestingly, the 
recent move to revise Massachusetts’ 
Goal-Setting Process raises some of the 
same issues that Don did in his article 
from 1993. 

Volume 6: Responding to the Dream 
Conference 

This volume focuses on writers’ 
reflections and use of the first 5 volumes 
of Adventures in Assessment. Dispersed 
throughout is commentary on the points 
of goal-setting and alternative assessment 
procedures and tools. Of relevance 
to this document’s purpose: 

“Affirmation for Pre-Goal Setting” by 
Anne Marie DeMartino: This letter is in 
response to Don Robishaw’s article from 
Volume 5, described above. 

Volume 7: Partnership Project 

This volume focuses on teachers as 
researchers in partnership with one 
another. Many of the articles reflect on 
on-going classroom events and progress 
of learners; of special interest is some 
focus on examining adult students’ self- 
esteem and motivation. 

“Taking Time to Talk: Students and 
Teachers Setting Goals” by Marty 
Tassi-Richardson and Deirdre McLaughlin: 



Procedures and tools for the goal-setting 
process developed in response to dissat- 
isfaction with those in use at that time 
at a particular program. 

“Self-Assessment for the Beginner: 

A Goals Oriented Approach” by Rudee 
Atlas and Dan Wilson: Although the focus 
is really about the on-going assessment 
process, this article provides some ideas 
for helping students identify goals and 
monitor achievements through self-assess- 
ment at lower levels and in a non-threat- 
ening format. 

“Bottoms Up: An Alternative Self-Directed 
Readiness Training Program" by Don 
Robishaw: A follow-up article to Don’s ear- 
lier one on the Pre-Goal Setting Process 
(Volumes 5); the program described is 
really for the on-going classroom, but still 
provides good ideas for making the goal- 
setting process more meaningful/valuable; 
some points could be adapted so that 
they could be incorporated into initial 
processes. 

“Authentic Assessment in the Workplace” 
by Debbie Tuler: Describes the start of 
the program; discusses initial assessment 
tools/procedures, initial selection process, 
curriculum/materials development; includes 
descriptions of an orientation and 
interview process. 

Volume 8 

Covers a range of articles that focus 
on the influence/affect of alternative 
assessment on topics from education 
reform to program management to learner 
involvement. 
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Volume 9 

The articles focus on examining how 
teachers (and students) assess continually 
in their classrooms. 

“Assessment in the ESOL Experience” by 
Elizabeth Santiago; Discusses alternative 
and commercial tools for placement and 
initial assessment. 

“Developing a Native Language Literacy 
Program” by Michelle Brown: Describes 
the quest by this program to find/develop 
an appropriate diagnostic tool for adults 
struggling with reading/writing skills in 
their own language. 

“Learning from Experience; The Native 
Language Literacy Screening Device” 
by Deborah Mercier-Cuenca: This tool 
is reviewed; ordering information is 
provided. 

Volume 10 

Volume 10 covers articles discussing 
ways that practitioners try to integrate 
both commercial and alternative assess- 
ments into their instruction. 

“The Haitian Multi-Service Center 
Experience” by Maria Kephallenou: 
Describes purposes of assessment from 
the different points of view of the partici- 
pants (student, teacher, counselor, etc.); 
gives overviews of each, and includes 
overviews of intake/placement, and 
exit/entrance criteria; examples of tools 
are provided. 

“Where’s the EGAP These Days?” by 
Martha Jean: This article briefly updates 



this tool, whose acronym stands for 
Educational Goals Assessment Package; 
it is designed to capture a range of 
student's interests, goals, and some skill 
ability; examples of some of the tools 
are provided; the article does not give 
contact information, but I am aware that 
the practitioner-writer presently works 
with the YALD program. 

“One Family Literacy Program’s 
Assessment Story” by Sylvia Greene et al: 
The article describes their initial assess- 
ment and in-take processes and provides 
examples of tools; on-going and exit 
assessment procedures are also 
overviewed. 

“The History of the BEST” by Moria 
Lucey; “Why I Think the BEST Isn't Good 
Enough” by Dulany Alexander; “The BEST 
is Workable, But It’s Not the Only Choice” 
by Barbara Lippel-Paul; “An AmeriCorps 
Volunteer’s First Impressions of the BEST” 
by Rachael Donnelly: I add these articles, 
whose titles are self-explanatory, for your 
information. 

“What Counts? Assessing Computer 
Skills” by Kenneth Tamarkin: This article 
describes a tool and process designed 
to gauge the appropriate placement level 
for students; tool is also used to help 
measure progress. 

“The TABE: Thoughts from an Inquiring 
Mind” by Cathy Coleman: The article 
reviews and critiques this test. 

Volume li 

These articles generally focus on 
how assessment is, or can be connected 
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to teaching, learning, and program 
accountability. 

“This is Only a Test” by Janet Isserlis: 
While this article does not focus on this 
document's purpose, Janet provides 
several web resources at the end 
including: ‘Let's Get Started: An initial 
assessment pack for adult literacy pro- 
grams' found at h ttp:/Awww.nald. ca/cl/aet- 
started/cover.htm ; in addition, I highly 
recommend Janet's website for us all: 

I find it to be one of the most 
comprehensive adult/non-traditional 
education websites around; found 
at httD://www.brown.edu/departments/ 
swearer center/literacy resources 

“The More Things Change, the More They 
Seem to Stay the Same” by Maria Elena 
Gonzalez: The article reviews various 
initial assessment tools used over the 
course of several cycles of a program; 
examples of several tools are provided. 

“Authentic and Learner-Centered 
Assessment in the Beginning ESOL 
Classroom” by Glen Cotton: While most 
of this article focuses on the 
practitioner-writer's research and 
subsequent classroom work, it does 
provide a brief description of processes 
for assessing learner purposes for learn- 
ing English. 

“Reflections on Meeting the Challenge 
of Assessment with Beginning Students” 
by Cheryl Gant: This article describes 
processes and tools (both commercial 
and alternative) that can be used to 
determine goals, and as a diagnostic 
at the lower ESOL level. 



New Wavs of Classroom Assessment 
by J.D. Brown, ed. as reviewed by Nancy 
Pendleton et al: Their review is quite 
positive and they note that the activities 
presented in the book include questions 
for pre-goal setting and self-assessment. 

Volume 12 

This volume focuses on teachers' 
and administrators' experiences working 
with standards-based reform initiatives. 

“To TABE or Not to TABE: One Agency's 
Options” By Bernie Driscoll: This article 
describes an agency's alternative place- 
ment tool for students of math; the 
math tool, which does not appear 
appropriate for very low levels, 
is provided. 

Volume 13: Meeting the Accountability 
Challenge 

All the articles focus on practitioners' 
and administrators' efforts to meet state 
and federal accountability demands. 

“Authentic Goal Setting with ABE 
Learners: Accountability for Programs or 
Process for Learning” by Sally Gabb: Sally 
provides a review of some of the articles 
focused on setting goals in Adventures in 
Assessment over the years; she also cri- 
tiques the goal-setting process as it is 
presently carried out in Massachusetts. 

“Quinsigamond Community College's Site- 
Specific Assessment' by Chris Hebert et al: 
The article describes the program's test 
for placing students at the SPL 7-10; the 
assessment is provided. 
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Volume 14 : Examining Performance 

All the articles in this volume discuss 
efforts to capture performance without 
the use of traditional or commercial tests. 

“Assessing Oral Communication 
at the Community Learning Center” 
by JoAnne Hartel and Mina Reddy: 

The article describes the agency’s test 
that can be used as a placement and 
diagnostic tool; examples of the tool 
are provided. 

“So what is a BROVI, Anyway? And how 
can it change your (assessing) life?” by 
Betty Stone and Vicki Halal: While the 
tool is generally for on-going assessment 
purposes, many of the activities described 
can be adapted for use as a placement 
or diagnostic; examples of the tool are 
provided. 



“A Writing Rubric to Assess ESL Student 
Performance” by Inaam Mansoor and 
Suzanne Grant: The REEP tool is 
described and the rubric is provided; 
as Massachusetts’ practitioners now 
know, the tool is also used for placement 
purposes. 

“Illuminating Understanding: Performance 
Assessment in Mathematics” by Tricia 
Donovan: This articles describes a 
process and rubric for working with 
performance assessment; while no 
specific tool or activity is presented, the 
approach is highly useful for practitioners 
in determining measures and diagnostic 
information when developing their own in- 
take and placement tools. 



Marie Cora is the Assessment Specialist for 
SABES and the editor of Adventures in 
Assessment . 



VOLUME 15 




PAGE 32 



ADVENTURES IN ASSESSMENT 



33 



U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 



NOTICE 



Reproduction Basis 



This document is covered by a signed "Reproduction Release (Blanket)" 
form (on file within the ERIC system), encompassing all or classes of 
documents from its source organization and, therefore, does not require a 
"Specific Document" Release form. 



This document is Federally- funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may be 
reproduced by ERIC without a signed Reproduction Release form (either 
"Specific Document" or "Blanket"). 





EFF-089 (1/2003) 




